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1. INTRODUCTION 

Over the years, the rapid developments in information and communication technologies (ICTs) have 
essentially influenced almost all aspects of human’s daily life. The overwhelming advancements of ICTs 
have created the so called “electronic” dimension, resulting in paradigm shifts related to business, 
governance and education. This digital means of communication also impacted on how government delivers 
information and services to the citizens by having e-government. One of the components in e-government is 
e-participation that allows active participation between citizens and the government. Several studies had been 
conducted to evaluate the accessibility and usability of e-government in general [1-4] but not the e- 
participation. 

In [5], defines citizen participation as “the process through which stakeholder influence and share 
control over priority setting, policymaking, resource allocations and access to public goods and services”. 
As indicated by [6], e-participation refers to the use of ICTs to support information provision and “top-down” 
engagement or “ground-up” efforts to empower citizens, civil society organizations and other democratically 
constituted groups to gain the support of their elected representatives. In [7], states that with the assistance of 
latest technology developments, e-participation aims to support active citizenship, increasing access to and 
availability of participation in order to promote fair and efficient society and government. 

E-participation is thought to be a fundamental component making a good e-democracy by helping 
people getting involved in politics and policy-making [8]. These citizen-government interactions likewise 
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empower new forms of collaboration where governments outline aggregate activity around particular issues 
and citizens will then initiate their own actions towards those issues [9, 10]. 

Despite the initiatives of many countries in adopting e-participation, there do not pay corresponding 
action in evaluating these efforts [11]. Similarly, [12] argue that the evaluation of both offline and online 
participation is still new and needs significantly further research. In [13], presumes that from their extensive 
literature on e-participation, there is a lack on both evaluation studies and established evaluation 
methodologies. In [14], likewise concur that the citizen online participation is the less evaluated concept of e- 
government websites. In [15] also agreed that evaluation is required to determine whether and to what extent 
an e-participation does contribute to enhancing democracy. 

This study discusses about the interaction experience of Malaysia e-participation module resides in 
e-government website. This paper commences with the application of the data collection method and 
concludes with a discussion on findings, limitations and further research. 


2. RESEARCH METHOD 

In this study, a concurrent think-aloud method was used in order to gain insights of users’ thought 
during the interaction with the e-participation website. Concurrent think-aloud method is one of the most 
common think-aloud protocols used in identifying usability problems [16]. Both [17, 18] applaud thinking 
aloud method as the most valuable usability evaluation method. This method permits the end user to thinking 
out loud while using system [19]. By verbalizing their thoughts, it enables the researchers to capture and 
understand how users view and interpret the system [20, 21]. A study conducted by [22] also proves that by 
thinking aloud, the participants can directly and efficiently verbalize any problems encountered from the 
interface design itself. 

A concurrent think-aloud method adapted from [19] was used in this study as it keep the interactions 
between researcher and participants to a minimum as not to interfere their feedback during the session. 
The only intervention occurred during the session was the phrase “keep-talking” when researcher noticed that 
participant was kept silenced for quite some time. The think-aloud protocol used in the study was piloted 
with two undergraduate college students who had no experience with this method before. 

The concurrent think-aloud session was carried out individually in a laboratory setting at Kolej Poly- 
Tech MARA Ipoh. Ten participants were selected on a voluntary basis among academic and non-academic 
staffs and a remuneration tokens were given to them after they finished their session. To begin with, 
each participant read the information sheet given to them and gave their written consent in participating in 
the study. They also completed a brief questionnaire on their demographics details. Following this, the think- 
aloud instructions were read out loud by the researcher in order to confirm their understanding. After that, 
a brief demonstration on the think-aloud method was presented to them by the researcher. This was done as a 
warm-up exercise so that the participant will be more comfortable especially in the presence of others [23]. 
Once done, the participant started to browse freely the Malaysia e-government portal 
(https://www.malaysia.gov.my). No particular tasks were given to the participant but they need to also 
browse the e-participation module contains in the website. The session lasted for about 15 minutes for every 
individual and each verbal utterance were recorded in by using audio recording device. Figure 1 underneath 
illustrated the basic think-aloud protocol that were used in the study. 
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Figure 1. Flow chart of the think-aloud protocol conducted in the study 
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3. RESULTS AND ANALYSIS 
3.1. Demography 

Demographics data were collected to provide a general description of the participants while 
questionnaire on Internet-use behavior was used to gain insight on their Internet use, knowledge and habits. 
A summary of the demographics data is presented in Table 1. 

Table | depicts the demographics profiles of the participants involved in this study. Altogether ten 
people participated in the think-aloud session (5 male and 5 female) with an average age of 33 years old 
(STDEV = 4.85). As stated by [24] it is possible to find about 75% of the usability problems by running only 
with small number of participants. Half of the participants held a master degree (50%) while one participant 
was a bachelor’s degree holder. 


Table 1. Demographics Profiles of the Participants 








Age Mean Minimum Maximum STDEV Total 
33 27 40 4.85 10 
Gender Male 5 (50%) Female 5 (50%) 
Highest Level of Total 
Education 
Diploma 4 (40%) 
Bachelor 1 (10%) 
Master 5 (50%) 





3.2. Internet-use Behavior 


The following Table 2 shows the descriptive statistics of Internet-use behavior of the participants 


involved in this study. 


Table 2. Descriptive Statistics of Internet-use Behavior 





Years of Internet use 


More than 5 years (100%) 





Internet connection at home 


No 
1 (10%) 

















Access to Internet* Total 
Home 9 (90%) 
Work 10 (100%) 
Library 7 (70%) 
Café 6 (60%) 
Other 2 (20%) 
Devices used* Total 
Desktop 8 (80%) 
Laptop 7 (70%) 
PDA - 
Mobile phone 10 (100%) 
Other 1 10%) 
Internet use frequency Total 
Weekly - 
2 —3 times a week - 
4 -6 times a week - 
Daily 5 (50%) 
2 —3 times a day - 
More than 3 times a day 5 (50%) 
Internet use duration Total 
Less than 5 minutes - 
6 -15 minutes 1 (10%) 
About 30 minutes 2 (20%) 
About | hour 2 (20%) 
Several hours 5 (50%) 
Purpose of using Internet* Total 
Email 10 (100%) 
Reading news 8 (80%) 
Online shopping 9 (90%) 
Social networking 10 (100%) 
Searching for study-related or 9 (90%) 
work-related information 
Surfing the web 9 (90%) 
Downloading or file sharing 9 (90%) 





* Participants were allowed to choose more than one option 
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From the Table 2, all of the participants (100%) have been using the Internet for more than five 
years and nine of them (90%) do actually have Internet connection at home. As all of them are working, 
their access to the Internet is 100% at work and all agreed that mobile phone is their chosen device to access 
the Internet. It is worth mentioning that in some of the questions, the participants had the liberty to choose 
more than one option as stated in the questionnaire. 


3.3. E-government-website-use Behavior 

The e-government-website use behavior of participants was also accessed in order to gain insight to 
their habits when it comes to interaction with e-government website. A summary of the analysis is presented 
in Table 3. 


Table 3. Descriptive Statistics of E-government-website use Behavior 
Yes No 
9 (90%) 1 (10%) 
E-government website visits frequency Total 
Weekly 9 (90%) 
2 —3 times a week - 
4 -6 times a week - 
Daily - 
2 —3 times a day - 
More than 3 times a day - 
E-government website use duration Total 
Less than 5 minutes 2 (20%) 
6 -15 minutes 6 (60%) 
About 30 minutes 1 (10%) 
About | hour - 
Several hours - 
Participation in any forum/poll/policy Yes No 
making in the e-government web site 1 (10%) 9 (90%) 





Ever visited e-government website before? 











Nine participants (90%) have visited e-government website before but only one (10%) participated 
in any forum/poll/policy making in the website. Nine participants (90%) reported of browsing the 
e-government website on weekly basis and majority of them (60%) spend about 6 to 15 minutes on 
e-government website. 


3.4. Analysis of the Think-aloud Data 

The think-aloud data were analyzed thematically to look for units of thought. This was done 
manually in an iterative manner to look for consistency of the code. As a result, 61 units of thought were 
identified and classified into main themes and sub themes. There are three main themes; content, design and 
information architecture and summarized as in Table 4. 


Table 4. Categories of Main Themes and Sub Themes 
# Main theme Sub themes 
14 Content Information content 
Language 
Relevancy 
Updated information 
2: Design Layout 
Selection of font, images 
Background/Foreground 














3. Information Architecture Navigation 
Scheme 
Link button/text 





Out of the three main themes, 59% (36 units) were referred to content while 16.4% (10 units) 
referred to information architecture. These findings are in line with the previous study conducted by [25] 
which saw that information or content is one of the crucial factor in designing e-government website. 
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4. CONCLUSION 

This study provides insights into users’ experiences with Malaysia e-government generally and 
e-participation module in particular. Due to the planning of not setting initial tasks to the participants, 
they figured out how to browse the website freely and reported genuinely verbal comments. 

Demographic results showed that all participants were experienced users of the Internet and thus 
having no problem in participating in this study. Think-aloud method proved to be one of the powerful tools 
to gain insights into users’ experience in interaction with a website. Findings of this study suggest that the 
content of a website plays an important role in engaging the user especially when it comes to e-participation 
as the content should be regularly updated. The number of problems encountered in this small scale study 
also show that people are still experiencing problems when using website. 

A limitation of this study is that it the sample size used in this study was sufficient for the purposes 
of this study and allowed reasonable conclusions to be drawn but it cannot be considered representative of all 
e-participation users in Malaysia. More representative samples within Malaysia can be considered for the 
next study to strengthen the current findings. Future research may also extend into other agencies that offer 
e-participation to cover a wider evaluation. 
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